Control Hierarchies for Critical Infrastructures in Smart Grid Using Reinforcement Learning and Metaheuristic Optimization

The objective of this work is to develop robust control framework for interdependent smart grid infrastructures comprising two critical infrastructures: 1) power distribution networks that are characterized by high penetration of distributed energy resources (DERs), and 2) DC-rail transportation systems in congested urban areas. The rising integration of DERs into the power grid is causing a paradigm shift in the power distribution network. Consequently, new control challenges for efficient and robust operation of the power grid have surfaced. For instance, the intermittency of renewable energy resources necessitates coordinated control of power flows, voltage regulators, and protection device settings of the online resources in the system. This can be achieved with the help of an active distribution network, equipped with a distribution management system that provides online solutions to control problems, in real-time, by having full or partial observability. Besides, the increasing electrification of other critical infrastructures, such as transportation and communication, necessitates controllers that can accommodate the over-arching control requirements of interdependent critical infrastructures. Present control approaches lack the amalgamation of active distribution management and the flexibility to accommodate other critical infrastructures. In this work two levels of control hierarchies, viz. 1) Primary Controller and 2) Secondary Controller, have been designed for the power distribution system. These controllers can provide active distribution management, which can be expanded for seamless integration of other critical infrastructures. Besides, a real-time simulation-in-the-loop testbed has been developed, so that both transient and steady state performance of the controllers can be evaluated simultaneously. This testbed has been developed using OPAL-RT and DSpace. The effectiveness of these controllers have been tested in three types of active distribution networks: 1) A modified IEEE 5-bus system equipped with a grid-connected microgrid that consists of two DERs, 2) A modified IEEE 13-bus system equipped with an islanded community microgrid (C_-Grid) comprising four DERs, and 3) A modified IEEE-30 bus system comprising five grid-connected distributed generations (DGs). The DERs used for this work are battery energy storage systems and photovoltaic systems. The Primary Controller has been designed for regulating voltage, frequency and current in the system, while maintaining stability of these parameters, in both grid-tied and islanded operating modes. These design approaches consider multiple points of coupling among the DERs, which is lacking in the existing literature that is primarily focused on single point of common coupling. Besides, this work shows a method of incorporating communication latency, which may exist between Primary and Secondary Controller, into the control design. This facilitates performance analysis of the primary controller, when it is subjected to communication latency, and accordingly develop mitigation techniques. The Secondary Controller has been designed using a reinforcement learning technique called Adaptive Critic Design (ACD). ACD can facilitate seamless integration of a power distribution network with other critical infrastructures. The ACD based algorithm functions as a distribution management system where its control objectives are to balance load and generation, to take preventive or corrective measures for mitigating failures and improving system resiliency, to minimize the cost of energy incurred by the loads by dispatching the DERs, and to maintain their state of health. Alongside DERs, the impact of DC-rail transportation on the power distribution network has been investigated here, with an objective of efficiently and economically reducing congestion in power substations. Hybrid energy storage systems, comprising battery, supercapacitor and flywheel, have been used as wayside energy storage technologies for this purpose. These storage technologies reduce congestion by supplying energy during acceleration and coasting of the trains, and replenish their energy by recapturing the regenerative braking energy during deceleration of the same. The trains consume/regenerate energy at a very high rate during acceleration/deceleration, thereby requiring storage technologies with both high energy and power densities. Hence, the three aforementioned storage technologies have been investigated for both standalone and hybrid operations, by considering system performance and resiliency alongside the percentage of energy recovered, without comprising the cost-recuperation over time. This has been achieved using a two-stage method, where the first stage comprises the development of detailed mathematical model of the rail system and the storage technologies. This mathematical model has been optimized using Genetic Algorithm, in order to obtain optimal combinations of type and size of the storage technologies for minimum cost, within the system constraints. In the second stage, a detailed simulation model has been developed by capturing all the dynamics of the transportation network, which could not be entirely represented in the mathematical model. The optimal sizes obtained from the first stage have been used in the second stage to evaluate their performance and accordingly adjust their values. Thus, the mathematical model provides initial values in a large search space, and these values are further tuned based on the results from simulation model

Quantitative Analysis of Regenerative Energy in Electric Rail Traction Systems

This paper aims at determining the influential factors affecting regenerative braking energy in DC rail transit systems. This has been achieved by quantitatively evaluating the dependence of regenerative energy on various parameters, such as vehicle dynamics, train scheduling, ground inclination and efficiency of the electrical devices. The recuperated power and energy have been presented by a mathematical model, comprising of a set of empirical forms, which allows for thorough analysis. A detailed simulation model of a typical DC-traction system has been developed to validate the developed empirical forms. The results verified the validity of the proposed mathematical model, and demonstrated the significance of the examined parameters on the regenerative power and energy of a train during a complete cycle. Knowledge of the parameters impacting the magnitude of regenerative energy should maximize the potential of harnessing regenerative energy

A Dual-Stage Modeling and Optimization Framework for Wayside Energy Storage in Electric Rail Transit Systems

In this paper, a dual-stage modeling and optimization framework has been developed to obtain an optimal combination and size of wayside energy storage systems (WESSs) for application in DC rail transportation. Energy storage technologies may consist of a standalone battery, a standalone supercapacitor, a standalone flywheel, or a combination of these. Results from the dual-stage modeling and optimization process have been utilized for deducing an application-specific composition of type and size of the WESSs. These applications consist of different percentages of energy saving due to regenerative braking, voltage regulation, peak demand reduction, estimated payback period, and system resiliency. In the first stage, sizes of the ESSs have been estimated using developed detailed mathematical models, and optimized using the Genetic Algorithm (GA). In the second stage, the respective sizes of ESSs are simulated by developing an all-inclusive model of the transit system, ESS and ESS management system (EMS) in MATLAB/Simulink. The mathematical modeling provides initial recommendations for the sizes from a large search space. However, the dynamic simulation contributes to the optimization by highlighting the transit system constraints and practical limitations of ESSs, which impose bounds on the maximum energy that can be captured from decelerating trains